Controlling Output Length in Neural Encoder-Decoders
نویسندگان
چکیده
Neural encoder-decoder models have shown great success in many sequence generation tasks. However, previous work has not investigated situations in which we would like to control the length of encoder-decoder outputs. This capability is crucial for applications such as text summarization, in which we have to generate concise summaries with a desired length. In this paper, we propose methods for controlling the output sequence length for neural encoder-decoder models: two decoding-based methods and two learning-based methods.1 Results show that our learning-based methods have the capability to control length without degrading summary quality in a summarization task.
منابع مشابه
Online Segment to Segment Neural Transduction
We introduce an online neural sequence to sequence model that learns to alternate between encoding and decoding segments of the input as it is read. By independently tracking the encoding and decoding representations our algorithm permits exact polynomial marginalization of the latent segmentation during training, and during decoding beam search is employed to find the best alignment path toget...
متن کاملEnd-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes. The alignment between the input and output sequences is established using an attention mechanism: the decoder emits each symbol based on a context cr...
متن کاملME-MD: An Effective Framework for Neural Machine Translation with Multiple Encoders and Decoders
The encoder-decoder neural framework is widely employed for Neural Machine Translation (NMT) with a single encoder to represent the source sentence and a single decoder to generate target words. The translation performance heavily relies on the representation ability of the encoder and the generation ability of the decoder. To further enhance NMT, we propose to extend the original encoderdecode...
متن کاملShell Mapping for Distribution matching
Distribution matching transforms independent and Bernoulli(1/2) distributed input bits into a sequence of output symbols with a desired distribution PA. Fixed-to-fixed length, invertible encoders and decoders based on shell mapping are presented. The matcher minimizes the informational divergence for a large class of distributions including the Maxwell-Boltzmann and geometric distributions. Fin...
متن کاملNeural Machine Translation by Jointly Learning to Align and Translate
Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder–decoders and encode a s...
متن کامل